Stochastic Covariance Compression

Covariance matrices are an effective way to capture global spread across local interest points in images. Often, these image descriptors are more compact, robust and informative than, for example, bags of visual words. However, they are symmetric and positive definite (SPD) and therefore live on a non-Euclidean Riemannian manifold, which gives rise to non-Euclidean metrics. These are slow to compute and can make the use of covariance features prohibitive in many settings, in particular k-nearest neighbors (kNN) classification. In this paper we present Stochastic Covariance Compression, an algorithm that compresses a data set of SPD matrices to a much smaller set with similar kNN characteristics. We show that we can reduce the data sets to 1/6 and in some cases even up to 1/50 of their original size, while approximately matching the test error of full kNN classification. In fact, because the compressed set is learned to perform well on kNN tasks, it sometimes even outperforms the original data set, while requiring only a fraction of the space and drastically reduced test-time computation.